Elastic auto-parallelization for stream processing applications based on a measured throughtput and congestion

ABSTRACT

A method for adjusting a data parallel region of a stream processing application includes measuring congestion of each parallel channel of the data parallel region, measuring a total throughput of all the parallel channels, and adjusting the number of parallel channels based on the current measured congestion and throughput.

BACKGROUND

1. Technical Field

The present disclosure relates to the data stream processing, and moreparticularly to data processing using parallel elastic operators.

2. Discussion of Related Art

As the world becomes more interconnected and instrumented, there is adeluge of data coming from various software and hardware sensors in theform of continuous streams. Examples can be found in several domains,such as financial markets, telecommunications, surveillance,manufacturing, and healthcare. In all of these domains, there is anincreasing need to gather, process, and analyze these data streams toextract insights as well as to detect emerging patterns and outliers.More importantly, this analysis often needs to be performed in nearreal-time.

Stream computing is a computational paradigm that enables carrying outof analytical tasks in an efficient and scalable manner. By taking theincoming data streams through a network of operators placed on a set ofdistributed hosts, stream computing provides an on-the-fly model ofprocessing. The frequent need for handling large volumes of live data inshort periods of time is a major characteristic of stream processingapplications. Thus, supporting high throughput processing is animportant requirement for streaming systems. It requires takingadvantage of multiple host machines to achieve scalability. Thisrequirement will become even more prominent with the ever increasingamounts of live data available for processing. The increasedaffordability of distributed and parallel computing, thanks to advancesin cloud computing and multi-core chip design, has made this problemtractable. However, this requires language and system level techniquesthat can effectively locate and efficiently exploit parallelizationopportunities in stream processing applications.

BRIEF SUMMARY

According to an exemplary embodiment of the invention, a method ofadjusting a data parallel region of a stream processing applicationincludes: measuring congestion of each parallel channel of the dataparallel region, measuring a total throughput of all the parallelchannels, and adjusting the number of parallel channels based on thecurrent measured congestion and throughput.

In an embodiment, each parallel channel comprises a computer operatorand all the computer operators perform a same function. In anembodiment, the adjusting includes adjusting the number of parallelchannels based on the current measured congestion and throughput andprevious historical measurements of the congestion and throughput.

According to an exemplary embodiment of the invention, a method ofadjusting a level of parallelism of an application operating on a systemincludes determining a current number of operators of the applicationfor operating on a same task, determining a congestion level of thesystem, increasing a current number of operators of the application ifthe congestion level exceeds a threshold, and decreasing the currentnumber of operators if the congestion level does not exceed thethreshold.

In an embodiment, the method includes determining whether the congestionlevel of the system has improved after increasing the current number ofoperators and decreasing the current number of operators if thecongestion level has not improved.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Exemplary embodiments of the invention can be understood in more detailfrom the following descriptions taken in conjunction with theaccompanying drawings in which:

FIG. 1( a) illustrates an exemplary streaming application.

FIG. 1( b) illustrates the exemplary streaming application with aparallel data region.

FIG. 2 illustrates exemplary code for an exemplary operator in theparallel data region.

FIG. 3 illustrates a method of adjusting a level of a parallelism in anapplication according to an exemplary embodiment of the invention.

FIG. 4 illustrates exemplary code that may be used to implement themethod of FIG. 3.

FIG. 5 illustrates exemplary code that may be used to migrate stateinformation among operators of the parallel data region.

FIG. 6 illustrates barriers used to prevent the level of parallelismfrom being adjusted until after the operators have migrated their stateinformation.

FIG. 7 illustrates an embodiment where a shared storage is used toexchange state information among the operators according to an exemplaryembodiment of the invention.

FIG. 8 illustrates a method of adjusting a level of parallelism in anapplication according to an exemplary embodiment of the invention.

FIG. 9 illustrates an example of a computer system capable ofimplementing methods and systems according to embodiments of thedisclosure.

DETAILED DESCRIPTION

Exemplary embodiments of the invention provide language and system leveltechniques that can effectively locate and efficiently exploitparallelization opportunities in stream processing applications.

A streaming application may be structured as a directed graph wherevertices are operators and edges are data streams. For example, a vertexor node of the graph performs an operation on input data and provides aresult of the operation to its output edge as an output data stream. Asystem can scale a streaming application by deciding how the applicationgraph will be mapped to a set of available hosts.

Auto-parallelization is an effective technique that can be used to scalea streaming application in a transparent manner. It involves detectingparallel regions in the application graph that can be replicated onmultiple hosts, such that each instance of the replicated region (e.g.,hereinafter referred to as a channel) handles a subset of the data flowto increase throughput. This form of data parallelization involvesdetecting parallel regions without direct involvement of the applicationdeveloper and applying runtime mechanisms to ensure safety. Theparallelized application should produce the same results as a sequentialapplication.

Transparent auto-parallelization that improves performance should havesome profitability mechanism. In a streaming data-parallel region,profitability involves determining the right degree of parallelism, thatis, the number of parallel channels that are to be used, withoutexplicit involvement of the application developer.

In at least one embodiment of the invention, auto-parallelization iselastic since it makes the profitability decision adaptive to runtimedynamics (e.g., changes in workload and availability of resources).

At least one embodiment of the invention achieves elasticauto-parallelization in in the presence of stateful operators. Astateful operator is an operator that performs a function that relies onstate information. For example, if the stateful operator is configuredto generate a tax return, many of its calculations may depend on thecurrent state of an individual's gross income. Thus, if the statefuloperator is duplicated so some of the calculations can be offloaded tothe duplicated operator, the current state of the gross income wouldneed to be communicated (migrated) to the duplicated operator. However,migration of such state information can be very burdensome. Thus, atleast one embodiment of the invention attempts to minimize the amount ofstate information that needs to migrated to a stateful operator tominimize time and space overheads.

At least one embodiment of the invention attempts to provide a controlalgorithm to determine the degree of parallelism that exhibits stability(i.e., does not oscillate the number of channels used), achieves goodaccuracy (i.e., finds the number of channels that maximizes thethroughput), has a short settling time (i.e., reaches a stable number ofchannels quickly), and avoids overshoot (i.e., does not use morechannels than necessary).

A stateful operator can be partitioned into partitioned statefuloperators, which store an independent state for each sub-streamidentified by a partitioning attribute. Examples of such operatorsinclude an operator to partition network traces by IP number, partitionfinancial streams by ticker, etc. Compile-time re-write techniques canbe developed to convert high-level user code into an equivalent versionthat use an API that shields application developers from the details ofstate migration.

In at least one embodiment of the invention, an operator is split usinga hashing algorithm, which minimizes the amount of state informationmigrated. Further, an embodiment of the invention relies on two localmetrics computed at the splitter: the congestion (e.g., a measure ofblocking time at the splitter) and the throughput. The congestion mayindicate how busy a particular operator is.

In an embodiment of the invention, the splitter is a run-time componentthat is co-located with the operator that is generating the stream to besplit for parallel processing. The control algorithm can work at thesplitter and use these metrics to adjust the number of channels to beused for processing the flow. The algorithm may include peeking up anddown in terms of the number of channels used based on changes inobserved metrics to address accuracy and overshoot; remembering pastperformances achieved at different operating points to addressstability; and rapid scaling to address settling time.

At least one embodiment of the invention provides an elasticauto-parallelization scheme that can handle stateful operators, worksacross multiple hosts, and is designed for general purpose streamprocessing applications.

Embodiments of the invention will be discussed with respect to theSystem S middleware and its programming language SPL for ease ofdiscussion. However, embodiments of the invention are not limited to anyparticular middleware or programming language.

SPL is a programming language used to develop stream processinglanguages. SPL applications are composed of operator instances connectedto each other via stream connections. An operator instance is a vertexin the application's data flow graph. An operator instance is arealization of an operator definition. For example, FIG. 1 shows aninstance of the TCPSource operator 101. In general, operators can havemany different instantiations, each using different stream types,parameters, or other configurations such as windows. Operator instancescan have zero or more input and output ports. Each output port generatesa uniquely named stream, which is a sequence of tuples. Connecting anoutput port to the input of an operator establishes a stream connection.A stream connection is an edge in an application's data flow graph.

Operators are implemented either directly in SPL or in a general purposeprogramming language. These implementations rely on an event driveninterface, which reacts to tuples arriving on operator input ports.Tuple processing generally involves updating some operator-local stateand producing result tuples that are sent out on the output ports.

System S is a distributed stream processing engine that can execute SPLapplications using a set of distributed hosts. System S can performvarious runtime tasks, such as data transport, scheduling,fault-tolerance, and security.

Auto-parallelization is the process of automatically discoveringdata-parallel regions in an application's flow graph which can beexploited at runtime. In addition to discovering these parallel regions,the compiler establishes certain properties required to activateappropriate runtime mechanisms that will ensure safety of theauto-parallelization. For instance, if a parallel region is determinedto be stateless, the runtime data splitting mechanism to be applied canbe round-robin, whereas if the region is partitioned stateful, the datasplitting is performed using a hash-based scheme.

FIG. 2 illustrates sample SPL code 200 for an exemplaryauto-parallelization process named OpMon. An instance of the TCPSourceoperator 101 is used to receive a stream that contains information aboutnetwork usage of different applications. This is followed by anAggregate operator instance 102, which computes minute-by-minute datausage information for each application, using an application identifier(e.g., appId) as the partitioning key. In FIG. 1( a) the aggregatedresults are taken through a Filter operator 103 to retain applicationswhose network usage is beyond a threshold. Finally, the end results aresent to a TCPSink operator instance 104.

In FIG. 1( b) the Aggregate operator 102 and filter operator 103 areduplicated to form a parallel region including parallel channels, whereeach parallel channel contains a copy of the aggregate operator and thefilter operation. For example, FIG. 1( b) illustrates ‘n’ parallelchannels including a first parallel channel (e.g., 102-1 and 103-1), asecond parallel channel (e.g., 102-2 and 103-2), and an n^(th) parallelchannel (e.g., 102-n and 103-n).

In this example, there is an additional operator TCPSink 104 thatfollows the parallel region. Further, since the operator TCPSink 104 maynot be able to tolerate out of order results, the parallel region needsto maintain the order of tuples at its output. This is achieved at themerger 106, which resides on the input port of the operator succeedingthe parallel region. The merger 106 performs a re-ordering operationusing sequence numbers which were assigned at the splitter 105 andcarried through the parallel region.

This parallel region contains Filter operators 103-1, 103-2, . . . ,103-n that can drop some of the tuples. This results in a selectivityvalue of at most 1. This may cause the merger 106 to block for longperiods of time, if the tuples for a given channel happen to get droppedwith a higher frequency than others. This is because during times of notuple arrival, the merger 106 cannot differentiate between tuples thattake a long time to arrive and tuples that will never arrive (dropped).Accordingly, in at least one embodiment of the invention, the parallelregion uses pulses, which are special markers periodically sent by thesplitter 105 and used by the merger 106 to avoid lengthy stalls.

At least one embodiment of the invention makes the profitabilitydecision (e.g., determines whether it is more profitable to increase ordecrease the level of parallelism) at runtime, where informationregarding workload and resource availability can be inferred. When anapplication starts its execution, the number of parallel channels is setto a predefined value (e.g., 1). A control algorithm is placed as thesplitter 105 periodically re-evaluates the number of channels to be usedbased on local run-time metrics it maintains. The control algorithm candecide to increase or decrease the number of channels used based or takeno action. When the number of channels to use changes, then a statemigration protocol can be executed if the parallel region is stateful.

For parallel regions that are partitioned stateful, changing the numberof parallel channel necessitates partial relocation of stateinformation. For instance, if the number of parallel channels increases,then the assignment of some of the partitions needs to move from theexisting parallel channels to the new parallel channels. Whenever suchchange of assignment happens at the splitter 105, the state informationassociated with the moved partitions has to be relocated as well. Inparticular, the newly added parallel channels need to borrow stateinformation of the partitions assigned to them from the existingparallel channels. Similarly, when existing channels are removed, thestate information associated with the partitions they were handling hasto be redistributed to the existing parallel channels.

In an exemplary embodiment, where each partition is owned by a singleparallel channel, partitions are assigned to parallel channels usingconsistent hashing to minimize the amount of state information movedduring migration.

As discussed above, the control algorithm performs a periodic evaluationto determine whether to increase or decrease the current level of aparallelism. In an exemplary embodiment of the invention, the controlalgorithm relies on two locally generated metrics, namely Congestion andThroughput.

Congestion is an indication of whether the splitter 105 observes anundue delay when sending tuples on a connection. Presence of congestionmay be an indication that more channels are needed to handle the currentload. Similarly, lack of congestion may be an indication that too manychannels are being used. For example, in an optimal situation, a channelis busy, but not overly burdened or underused. Temporal changes in thecongestion value can indicate changes in the workload availability. Forexample, if several parallel channels have a moderate congestion (i.e.,busy, but not overly burdened or underused) at a first time and theneach of the channels have a minimal congestion (i.e., underused) at asecond time, one can infer that the overall workload has decreasedconsiderably.

The congestion can be computed by using non-blocking I/O fortransferring tuples. For example, a blocking period can be calculatedfrom a difference between a first time a send call provides anotification that the call would block and a second time the send callprovides a notification that room is available. The congestion value canthen be based on the measured blocking period as it compares tocomparison thresholds or ranges. For example, if the blocking period ismore than one second it could be deemed a congestion value of 1 (i.e.,indicating excessively congested), if the blocking period is between 0.5second and 1 second it could be deemed a congestion value of 0.5 (i.e.,moderately congested), and if the blocking period is less than 0.5seconds, it could be deemed a congestion value of 0 (i.e., minimalcongestion). Please note that the above-provided comparisonthresholds/ranges are merely examples as the calculation of congestionis not limited to any particular threshold/range.

Throughput is the number of tuples processed per second over the lastadaptation period. For example, at a given time when the number ofchannels has been increased or decreased (e.g., a particular operatingpoint), a current throughput compared to a previous throughput canindicate whether throughput has improved. Further, temporal changes inthroughput can indicate changes in the workload. For example, ifthroughput typically averages 10,000 tuples per second and then beginsaveraging 1,000 tuples per second, it can be inferred that the workloadhas been decreased considerably.

At least one embodiment of the invention increases the level ofparallelism if congestion is present unless that increase was alreadytried and it was not found to be beneficial and decreases the level ofparallelism if no congestion is present unless that decrease was alreadytried and it was not found to be beneficial.

FIG. 3 illustrates a method for controlling the number of parallelchannels according to an exemplary embodiment of the inventive.Initially, the method begins with a certain number of channels N_(L)(e.g., 2) (S301). Next, the method forgets obsolete information (S302).For example, if it was previously determined that a current congestionlevel indicates that the workload has changed, the prior snapshots ofthe previous congestion levels are deleted. Similarly, if a currentthroughput level indicates that the workload has changed, the priorsnapshots of the previous throughput levels are deleted.

Next, the method determines whether the current number of channels inuse (e.g., 2) are congested (e.g., being overworked) (S303). If thecurrent number of channels in use are not congested (e.g., underutilized), the method determines whether the previous lower number ofchannels (e.g., 1) was congested (e.g., overworked) (S304).

If the previous lower number of channels was not congested (e.g., underutilized), the method decreases the level of parallelism by reverting tothe previous lower number of channels (e.g., 1) (S309). If the previouslower number of channels was also congested, the current level ofparallelism is maintained since using a lesser number of channels willnot improve congestion.

If the current number of channels in use was congested, the methoddetermines whether this congestion is remote congestion (S305). Remotecongestion means that the congestion is attributed primarily due to anapplication/process that is external to the parallel operators. If it isdetermined that the congestion is attributed to the remote congestion,the method decreases the level of parallelism by reverting to theprevious lower number of channels (S306). This step avoids the casewhere the number of channels is continuously increased due to thecontinued presence of congestion, yet the throughput does not improve.Since the only congestion left is due to remote congestion, thestreaming application has hit a scalability limit. For example, theoriginal bottleneck introduced by the streaming application has beenremoved due to parallelization and the bottleneck has moved to anon-parallelization portion (e.g., the source or sink of some statefuloperator) of the application.

If it is determined that the congestion is not attributed to remotecongestion, the method determines whether the throughput at a nexthigher number of channels is higher than the current amount ofthroughput or unknown (S307). For example, if the next higher number ofchannels (e.g., 3) was never used or the corresponding throughput databecame obsolete, this data would not be present. If the throughput atthe next higher number of channels is higher than the current amount ofthroughput or unknown, the method increases the level of parallelism byincreasing the number of channels (e.g., to 3) (S308). However, if thethroughput at the next higher number of channels is the same or lower,the method maintains the current level of parallelism.

In cases where the available resources (e.g., execution contexts such ashosts and cores) and the cost of the parallel region are both high, theoptimal number of channels can be high as well. Thus, in practice itcould take a long time for the above-described control algorithmillustrated in FIG. 3 to reach this number. This is due to theone-channel-at-a-time nature of the algorithm. In an alternateembodiment of the algorithm of FIG. 3, rapid scaling is introduced. Inrapid scaling, rather than operating (e.g., increasing/decreasing)one-channel-at-a-time, one can operate (e.g., increase/decrease)one-level-at-a-time, where a super-linear mapping is defined between thenumber of levels and channels. For example, one level can correspond tomore than one channel. For example, in this way steps S306, S308, andS309, can increase/decrease the current number of channels by more thanone. Rather, than using the number of channels as the operating point,the method of FIG. 3 can use a level, which is mapped to the number ofchannels via function N_(L). In an exemplary embodiment of theinvention, the function N_(L) is defined by Equation 1 as follows:

N _(L)=[0.5+2^(0.5*(L+1))]  [Equation 1].

If the above function N_(L) is used, for increasing level L valuesstarting at 0 (1 by 1) results in the following series of number ofchannels: {1, 2, 3, 4, 6, 8, 11, 16, 23, 32, . . . }. Thus, if themethod of FIG. 3 were to determine that is necessary to increase thelevel of the parallelism in five steps, the first three steps wouldincrease the number of channels by 1 and the next two steps wouldincrease the number of channels by 2. However, embodiments of theinvention are not limited to any particular function N_(L) since it ispossible to use other functions that follow a steeper or less steepcurve depending on the maximum number of channels and the settling timerequirements.

In an exemplary embodiment, the method of FIG. 3 is implemented as acomputer program that stores three state variables. FIG. 4 illustratesan example of the program 400 including initialization code 401 toinitialize the state variables, first code 402 to update the number ofchannels, and second code 403 to detect workload changes. The first codetakes as input parameters the current throughput (T) and the currentcongestion status (C).

The first state variable P is the current adaptation period. The secondstate variable L represents the current level. The third state variableis an array that stores information for each level, namely the lastadaptation period P_(i) during which the algorithm was at this level,whether congestion, denoted C_(i) was observed the last time thealgorithm was at this level, the throughput T_(i) ^(−|) observed thelast time the algorithm was at this level, and the throughput T₁ ^(|−)observed during the first of the periods the last time the algorithmstayed consecutive periods at this level. The parameter L* denotes themaximum number of levels.

The program 400 has a global parameter called change sensitivity,denoted by α, which determines what a significant change means and takesa value in the range [0, 1]. A value of 1 means the algorithm is verysensitive to small changes in the throughput. For instance, a minorimprovement in throughput will be sufficient to increase the number ofchannels if the sensitivity is high. In an exemplary embodiment, allchanges in throughput are normalized against the ideal throughput for asingle channel in a linearly scaling system.

The second code 403 includes code 403-A to detect load changes based oncongestion status. If the current level and the last level are the same,yet the congestion status has changed, this is taken as an indication ofload change (e.g., load increased if there is congestion currently, loaddecreased otherwise). If the current level is lower than the last one,yet the congestion has disappeared, this is taken as a load decrease. Ifthe current level is higher than the last one, yet the congestion hasappeared, this is taken as a load increase. The second code includescode 403-B to detect load changes based on throughput. If the currentlevel and the last level are the same, yet there is a significant changein the throughput, this is taken as an indication of load change (e.g.,load increase if the current throughput is higher, load decreaseotherwise). Change sensitivity is used to detect significant changerelative to the ideal change in a linearly scaling system. If thecurrent level is lower than the last one, yet the throughput hasincreased, this is taken as a load increase. If the current level ishigher than the last one, yet the throughput has decreased, this istaken as a load decrease.

A state management application programmer interface (API) can bedeveloped for use by a compiler when generating code for an elasticoperator to enable elastic parallelism. The API may include apartitioned state clause to specify the list of state variables to bemaintained on a per-partition basis and a partitionBy parameter tospecify the partitioning attribute to be used for the partitioned state.For operators that are developed in general purpose programminglanguages, such as C++ and Java, the API may be provided as a nativeinterface.

As discussed previously, when the number of channels is increased it maybe necessary to migrate state information from an existing operatorcorresponding to the lower number of channels to a new operator that iscreated by increasing the number of channels. A migration protocol maybe executed for a parallel region (e.g., the set of operatorscorresponding to the current number of channels) in response to thedecisions made at the splitter 105 by the control algorithm (see methodFIG. 3). When the control algorithm updates the number of channels, italso updates the data partitioning function it uses to distribute thepartitions among the parallel channels and initiates the migrationprotocol. The migration is only needed for the case of partitionedstateful parallel regions. The migration protocol is initiated bysending a migration pulse from the splitter 105 to all parallelchannels. When an operator in a parallel channel receives a migrationpulse, it first forwards the pulse downstream and then starts executingthe per-operator migration protocol. This makes it possible to executemigration of state information between replicas of multiple operators inparallel, in case the parallel region contains more than one partitionedstateful operator.

An exemplary embodiment of a migrate routine 500 illustrated in FIG. 5provides pseudo-code for the migration protocol executed by an operator.There are four input parameters to the routine. The first is the indexof the operator's parallel channel, denoted by i. The second is the newoperating point in terms of the number of channels, denoted by N. Thethird is the state kept locally at this operator, which consists of alist of managed stores, denoted by S_(i) where s_(i) ^(k)εS_(i) denotesone of the stores. The last is the data partitioning function generator,which generates a data partitioning function given the number ofparallel channels, denoted by H. The protocol has two phases, namely thelend phase and the borrow phase.

In the lend phase, the items that do not belong to the current operatorafter the data partitioning function has been updated based on the newnumber of channels, are collected into a package Δ_(i→j) ^(k), whichrepresents the set of data items in s_(k) ^(i) that needs to migratefrom the operator replica running on the i^(th) channel to replicaterunning on the j^(th) channel. These items are removed from thein-memory store s_(i) ^(k). The resulting packages are stored on abacking store and then a vertical barrier is performed across replicasof the operator. This ensures that all replicas complete the lend phasebefore the borrow phase starts.

In the borrow phase, packages in the backing store that are destined tothe current operator replica are retrieved and the in-memory stores areupdated. For instance, items in package Δ_(j→i) ^(k) are added to thestore s_(i) ^(k). A vertical barrier is performed to ensure all replicashave completed the borrow phase. Once complete, a horizontal barrier isperformed, in order to ensure that the splitter does not start sendingtuples before the migration is complete. This barrier is performedacross the master operator replicas (at index 0) and the splitter.

FIG. 6 illustrates a parallel region with 2 operators (e.g., 102 and103) and N parallel channels. During a vertical barrier each operatorsynchronizes with its N−1 replicas, whereas during a horizontal barrierthe 2 operators on channel 0 synchronize with the splitter. Thisimplementation of the migration protocol works across multiple machinesand does not rely on shared memory. It at least one embodiment of theinvention, the implementation makes use of a back-end database for stateinformation movement and synchronization. Alternative implementationsare possible (e.g., sockets or a message passing interface MPI). WhileFIG. 6 illustrates a parallel region with 2 operators, in alternateembodiments, the parallel region could have a single operator or morethan 2 operators.

Data partitioning is performed at the splitter 105 for partitionedstateful regions. The partitioning function of the splitter 105 needs beupdated when the number of parallel channels changes. The choice of thepartitioning function impacts the cost of the migration, as it changesthe amount of migrated state information. The partitioning function canapply a hash function on the partitioning attributes and mod the resultbased on the number of channels. However, this data partitioningfunction may result in massive state information migrations and movementof some partitions across channels that are present both before andafter the migration.

Thus, a data partitioning function according to at least one embodimentof the invention provides good balance and monotonicity. Balance ensuresthat the partitions are uniformly distributed across channels, achievinggood load balance. Monotonicity ensures that partitions are not movedacross channels that are present before and after the migration.Consistent hashing is a technique that provides these properties.Consistent hashing maps each data item to a point on a 128-bit ring inuniformly random fashion. Similarly, each channel is also mapped to thesame ring, but rather than to a single point, each channel is mapped tomultiple points on the ring (using multiple hash functions). A data itemis assigned to the channel that is closest to it on the ring. As aresult of this scheme, when a new channel is inserted, it borrows dataitems from multiple of the existing channels. Similarly, when a channelis removed, its data items are distributed over multiple of the existingchannels. Consistent hashing ensures that on average MIN partitions aremoved when the Nth channel is inserted or removed from a system with Mpartitions. Consistent hashing can be implemented in O(1) time bydividing the ring into segments, yet it is slightly more costly tocompute as compared to a simple hashing scheme. However, it minimizesthe amount of state information to be moved during migration.

While the inventive concept has been described with respect to aparallel region including an aggregate operator and a filter operator,the inventive concept is not limited to any particular set of operatorsor functions thereof. For example, FIG. 7 shows a generic example, wherethe first parallel channel includes two operators F₁ and τ₁, the secondparallel channel includes two operators F₂ and Σ₂ the third parallelchannel includes operators F₃ and Σ₃, and the n^(th) operator includestwo operators FN and ΣN. As discussed above, a parallel region may onlyinclude a single operator (e.g., only F₁-FN). FIG. 7 additionallyillustrates a storage 700 that is used to store state information froman operator that needs to be migrated to another operator in a parallelchannel. The operators can store and retrieve state information from theshared storage 700. If the number of channels is to change, the existingoperators need to store their state information in the storage 700beforehand. If the number of channels is then increased, the newoperator can retrieve any necessary state information from the storage700. If the number of channels is instead decreased, then the remainingoperators can retrieve state information from the storage 700 that wasbeing retained by the deleted operator. A barrier can be used to delaythe deletion of an operator until it has had time to store its stateinformation in the storage 700. A barrier can also be used to delaypropagation of data to a set of the operators (F₁-FN) until after theyhave retrieved the necessary state information from the storage 700.

FIG. 8 illustrates a method for dynamically adjusting the number ofparallel channels for a data parallel region of a stream processingapplication according to an exemplary embodiment of the invention. Themethod includes: measuring the congestion of each parallel channel at acurrent period and maintaining a history (S801), measuring the totalthroughput of all parallel channels at the current period andmaintaining a history (S802), and adjusting the number of channels basedon the current and historical measurements of congestion and throughput(S803).

In at least one embodiment, the adjusting of the number of channelsincludes a splitter communicating to existing parallel channels to startstate migration, each channel determining what state information itneeds to retain and sending that state information to other channelsthat need this state information, and each channel receiving the stateinformation that it needs.

In an exemplary embodiment, the adjusting of the number of channelsincludes increasing the number of channels only if there is currentcongestion that is not remote congestion, and there is no historicalthroughput data for the higher number of channels.

In an exemplary embodiment, the adjusting of the number of channelsincludes increasing the number of channels only if there is currentcongestion that is not remote, and the historical throughput indicatesthat it is profitable to increase the number of channels. For example,if throughput in the past has improved by increasing the current numberof channels, then it makes sense to once again increase the currentnumber of channels.

In an exemplary embodiment, the adjusting of the number of channelsincludes decreasing the number of channels if there is currentcongestion and the congestion is remote. For example, if the number ofchannels was previously increased, and it could not remove the currentcongestion because it is remote congestion, the number of channels canbe decreased.

In an exemplary embodiment, the adjusting of the number of channelsincludes decreasing the number of channels only if there is nocongestion, and the historical data indicates that there was congestionat the lower number of channels.

In an exemplary embodiment, the adjusting of the number of channelsmaintains the same number of channels if there is no current congestion,and the historical data indicates that there was congestion at the lowernumber of channels.

In an exemplary embodiment, the adjusting of the number of channelsmaintains the same number of channels if there is current congestionthat is not remote congestion, and the historical data indicates that itis not profitable at the higher number of channels. Thus, even thoughthe congestion is not remote, since in the past increasing the number ofchannels has not improved this local congestion, the number of channelscan be kept at their current value.

The method can adapt to work load changes by selectively forgetting thehistorical measurements of congestion and/or throughput that have becomeobsolete. For example, if the measurements were taken at one workloadlevel, and the workload has since changed, the old measurements shouldbe disregarded. In an exemplary embodiment, if the workload hasincreased (e.g., by more than a certain threshold), the measurements forthe historical congestion and throughput from the higher number ofchannels is discarded. In another exemplary embodiment, if the workloadhas decreased (e.g., by more than a certain threshold), the measurementsfor the historical congestion and throughput from the lower number ofchannels is discarded.

The above described operators may be computer software threads orcomputer processes.

FIG. 9 illustrates an example of a computer system, which may executeany of the above-described operators, methods or computer programs,according to exemplary embodiments of the invention. For example, theoperators of FIGS. 1, 6, and 7, the code of FIGS. 2, 4, and 5, and themethods of FIGS. 3 and 8 may be implemented in the form of a softwareapplication running on the computer system. Further, portions of themethods may be executed on one such computer system, while the otherportions are executed on one or more other such computer systems.Examples of the computer system include a mainframe, personal computer(PC), a handheld computer, a server, etc. The software application maybe stored on a computer readable media (such as hard disk drive memory1008) locally accessible by the computer system and accessible via ahard wired or wireless connection to a satellite or a network, forexample, a local area network, or the Internet, etc.

The computer system referred to generally as system 1000 may include,for example, a central processing unit (CPU) 1001, random access memory(RAM) 1004, a printer interface 1010, a display unit 1011, a local areanetwork (LAN) data transmission controller 1005, a LAN interface 1006, anetwork controller 1003, an internal bus 1002, and one or more inputdevices 1009, for example, a keyboard, mouse etc. As shown, the system1000 may be connected to a data storage device, for example, a hard disk1008 (e.g., a digital video recorder), via a link 1007. CPU 1001 may bethe computer processor that performs the above described methods.

As will be appreciated by one skilled in the art, aspects of the presentdisclosure may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present disclosure may take theform of an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present disclosure may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readablestorage medium. A computer readable storage medium may be, for example,but not limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, or device, or any suitablecombination of the foregoing. More specific examples (a non-exhaustivelist) of the computer readable storage medium would include thefollowing: an electrical connection having one or more wires, a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), an optical fiber, a portable compact disc read-onlymemory (CD-ROM), an optical storage device, a magnetic storage device,or any suitable combination of the foregoing. In the context of thisdocument, a computer readable storage medium may be any tangible mediumthat can contain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device. Program codeembodied on a computer readable medium may be transmitted using anyappropriate medium, including but not limited to wireless, wireline,optical fiber cable, RF, etc., or any suitable combination of theforegoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present disclosure are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

1. A method for adjusting a data parallel region of a stream processingapplication, the method comprising: measuring congestion of eachparallel channel of the data parallel region; measuring a totalthroughput of all the parallel channels; and adjusting the number ofparallel channels based on the current measured congestion andthroughput, wherein each parallel channel comprises a same sequence ofat least one software operators that perform operations on a stream ofdata received from a same input software operator, and wherein themeasured congestion of a given channel indicates a utilization level ofthe channel.
 2. (canceled)
 3. The method of claim 1, wherein theadjusting comprises adjusting the number of parallel channels based onthe current measured congestion and throughput and previous historicalmeasurements of the congestion and throughput.
 4. The method of claim 3,wherein the adjusting comprises a splitter routine of the streamprocessing application indicating to each operator to begin migration ofstate information.
 5. The method of claim 4, wherein the adjustingfurther comprises each operator performing: determining what stateinformation presently being maintained by the operator will be needed bythe other operators; and sending the determined state information to theother operators.
 6. The method of claim 5, wherein the sending compriseseach operator storing the determined state information in a storage thatis accessible by all the operators.
 7. The method of claim 3, whereinthe adjusting comprises: increasing the number of channels if thecurrent congestion is above a threshold and it is not primarilyattributed to an application that is external to the operators; anddecreasing the number of channels if the current congestion is above thethreshold and it is primarily attributed to an application that isexternal to the operators.
 8. The method of claim 7, wherein theincreasing only occurs when the historical measurements of throughput donot include measurements for the higher number of channels.
 9. Themethod of claim 7, wherein the increasing only occurs when thehistorical measurements of throughput indicate it is profitable toperform the increase.
 10. The method of claim 3, wherein the adjustingcomprises decreasing the number of channels if the current congestion isbelow a threshold and the historical measurements of congestion do notinclude measurements for the lower number of channels.
 11. The method ofclaim 3, wherein the adjusting comprises maintaining the same number ofchannels if the current congestion is below a threshold and thehistorical measurements of congestion indicate that the lower number ofchannels have a congestion above the threshold.
 12. The method of claim3, wherein the adjusting comprises maintaining the same number ofchannels if the current congestion is above a threshold, it is notprimarily attributed to an application that is external to theoperators, and the historical measurements indicates that it is notprofitable to increase the number of channels.
 13. The method of claim3, wherein the adjusting comprises: discarding some of the historicalmeasurements of congestion and throughput when a current measure ofworkload differs from a previous measure of workload; and adjusting thenumber of parallel channels based on the current measured congestion andthroughput and the remaining historical measurements.
 14. The method ofclaim 13, wherein the historical measurements of congestion andthroughput from a higher number of channels are discarded when thecurrent measure of workload is higher than the previous measure.
 15. Themethod of claim 13, wherein the historical measurements of congestionand throughput from a lower number of channels are discarded when thecurrent measure of workload is lower than the previous measure. 16.(canceled)
 17. A method of adjusting a level of parallelism of anapplication operating on a system, the method comprising: determining acurrent number of software operators of the application for executing asame task on a stream of data; determining a congestion level of thesystem using the current number of software operators, wherein thecongestion level is based on a utilization level of each operator;increasing the current number of software operators of the applicationif the congestion level exceeds a threshold; and decreasing the currentnumber of software operators if the congestion level does not exceed thethreshold.
 18. The method of claim 17, further comprises: determiningwhether the congestion level of the system has improved after increasingthe current number of operators; and decreasing the current number ofoperators if the congestion level has not improved.
 19. The method ofclaim 17, wherein the operators are software threads.
 20. The method ofclaim 17, wherein the operators are distinct computer processes.
 21. Themethod of claim 1, wherein measuring the congestion for a given channelcomprises calculating a blocking period from a difference between afirst time a send call across the channel provides a notification thatthe call will block and a second time the send call across the channelprovides a notification that room is available.
 22. The method of claim17, wherein the increasing only occurs if the congestion level is notprimarily attributed to an application that is external to the operatorsand the method further comprises decreasing the current number ofsoftware operators if the current congestion level exceeds the thresholdand is primarily attributed to an application that is external to theoperators.